ADAPTIVE FILTERING IN REPRODUCING KERNEL HILBERT SPACES By WEIFENG LIU A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ADAPTIVE FILTERING IN REPRODUCING KERNEL HILBERT SPACES By Weifeng Liu December 2008 Chair: Jose C. Principe Major: Electrical and Computer Engineering The theory of linear adaptive filters has reached maturity, unlike the field of nonlinear adaptive filters. Although nonlinear adaptive filters are very useful in nonlinear and nonstationary signal processing, complexity and non-convexity issues limit existing algorithms like Volterra series, time-lagged feedforward networks and Bayesian filtering in an online scenario. Kernel methods are also nonlinear methods and their solid mathematical foundation and experimental successes are making them very popular in recent years, but most of the algorithms use block adaptation and are computationally very expensive using a large Gram matrix of dimensionality given by the number of data points; therefore computationally efficient online algorithms are very much needed for their useful flexibility in design. This work developed systematically for the first time a class of on-line learning algorithms in reproducing kernel Hilbert spaces (RKHS). The reproducing kernel Hilbert space provides an elegant means of obtaining nonlinear extensions of linear algorithms expressed in terms of inner products using the so-called kernel trick. We presented kernel extensions for three well-known adaptive filtering methods, namely the least-mean-square, the affine-projection-algorithms and the recursive-least-squares, studied their properties and validated them in real applications. We focused on revealing the unique structures of the linear adaptive filters and demonstrated how the nonlinear extensions are derived. These algorithms are universal